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percent  of  all  hybrid  groups  solved  die  posttask,  compared  to  the  83  percent  of  the  DEI 
group.  One  hundred  percent  of  all  hybrid  groups  said  the  hybrid  aids  were  easier  to  us 
and  understand  than  existing  technical  material.  In  a  test  on  system  knowledge,  th 
SS/BRC  group  was  superior  when  compared  to  the  other  groups.  Combining  the  result 
with  the  user  preference  for  the  SS/BRC  format,  it  was  concluded  that  the  SS/BRC  wa 
the  most  effective  EHJPA. 
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FOREWORD 

This  research  and  development  was  guided  by  Navy  Decision  Coordinating  Paper 
(NDCP)  Z0828-PN  (Enlisted  Personnel  Individualized  Career  System  (EPICS)  (formerly 
entitled  Performance  Aids  Test  and  Evaluation)),  under  the  sponsorship  of  the  Deputy 
Chief  of  Naval  Operations  for  Manpower,  Personnel,  and  Training  (OP-01).  The  objectives 
of  the  NDCP  are  to  define  the  state  of  the  art  in  job  performance  aid  (JPA)  technology, 
develop  a  conceptual  model  for  an  integrated  JPA-based  personnel  system  including  cost 
benefits  and  trade-off  analysis,  test  the  3PA  concept,  and  quantify  performance  incre¬ 
ments  and  costs  benefits  obtainable  for  various  applications. 

This  report  is  the  seventh  in  a  series  of  NAVPERSRANDCEN  reports  dealing  with 
3 PA  technology  development.  Previous  reports  described  the  status  of  3PA  technology 
.  CTR  77-33),  a  personnel  system  concept  emphasizing  the  use  of  3PAs  (TN  78-6),  3PA 
research  and  state  of  the  art  (TR  78-26),  a  3PA  selection  algorithm  for  an  integrated 
personnel  system  (TN  79-1),  development  of  hybrid  and  enriched  hybrid  3PAs  (TR  79-25), 
and  the  development  and  testing  of  a  troubleshooting  aid  for  digital  systems.  The  purpose 
of  the  present  effort  was  to  evaluate  hybrid  3PAs  in  an  operational  environment.  Results 
are  intended  for  the  3PA  and  technical  documentation  community. 

Appreciation  is  expressed  to  the  personnel  of  the  AN/AWG-10  Missile  Control  System 
School  at  NAS  Oceana,  Virginia  for  their  assistance  in  conducting  this  study. 


3AMES  F.  KELLY,  3R. 
Commanding  Officer 


3AMES  W.  TWEEDDALE 
Technical  Director 
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Prior  research  in  job  performance  aid  (JPA)  formats  for  troubleshooting  has  focused 
pn  the  development  of  deductive  JPAs  for  the  fully  trained  and  experienced  technician 
and  dfcrectlve  JPAs  for  the  inexperienced  technician.  Neither  format  is  satisfactory  for 
an  individual  at  an  inter me<fiate  skill  level. 

The  hybrid  JPA  presents  troubleshooting  information  In  both  directive  and  deductive 
formats.  The  intended  purpose  of  tine  hybrid  JPA  is  to  allow  an  inexperienced  technician 
to  troubleshoot  a  given  set  of  symptoms  correctly  using  the  directive  part  of  the  hybrid. 
At  the  same  time,  by  examining  the  deductive  part  of  the  hybrid,  the  apprentice 
technician  can  begin  to  incorporate  standard  technical  data,  such  as  functional  flow 
digrams  or  schematics,  in  his  troubleshooting  approach.  Over  time,  the  technician  should 
gradually  transition  from  the  directive  to  the  deductive  dement  and  begin  to  develop 
troubleshooting  strategies. 

In  an  earlier  effort,  the  potential  of  "enriching  the  hybrid  was  evaluated.  Enrich¬ 
ment  Is  a  method  of  adding  job  relevant  information  that  should  facilitate  the  transition 
between  directive  and  deductive  formats  by  providing  the  technician  with  a  functional 
understanding  of  the  system.  Various  conceptual  formats  far  the  enriched  hybrid  JPA 
(EHJPA)  were  defined,  but  none  were  evaluated  to  an  operations!  setting  to  determine 
whether  a  technician  could  transition  from  the  directive  to  the  deductive  or  if  the 
enrichment  information  contained  to  the  EHJPA  increased  the  technician's  functional 
understanding  of  the  system. 

In  addition  to  using  the  EHJPA  to  facilitate  an  inexperienced  technician’s  trouble¬ 
shooting  abilities,  the  deductive  element  of  the  EHJPA  can  be  used  by  the  more 
experienced  technicians  for  troubleshooting,  to  the  instructional  setting,  the  EHJPA 
would  allow  instructors  to  point  out  to  the  students  the  relationship  between  the  directive 
elements  and  the  development  of  a  deductive  troubleshooting  strategy. 

Purpose 

The  purpose  of  this  study  was  to  develop  and  evaluate  alternative  hybrid  JPA  formats 
to  determine  if  (1)  inexperienced  technicians  can  transition  to  a  deductive-only  JPA  after 
using  a  hybrid  JPA  to  facilitate  the  formation  of  troubleshooting  strategies  and  (2) 
placement  of  additional  information  (enrichment)  to  a  JPA  is  an  effective  technique  for 
increasing  technicians’  system  knowledge.  The  EHJPA  to  an  integral  component  of  the 
enlisted  personnel  individualized  career  system  EHCSK  In  EnCS,  technicians  need 
troubleshooting  3PAs  to  supplement  the  formal  training  fat  is  deferred  and.  distributed 
over  an  individual's  career. 
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1.  All  of  the  hybrid  groups  required  less  time  on  the  troubleshooting  posttask  than 
did  the  DED  group.  However,  only  the  TS/IC  group  completed  the  task  significantly 
faster  than  did  the  DEO  group. 

2.  For  the  posttask,  fbe  solution  rate  for  the  hybrid  pups  was  100  percent, 
compared  to  IS  percent  for  the  DED  control  group. 

X  Between  die  pre>  ami  posttests  on  system  knowledge,  the  SS/BRC  group 
improved  by  38  percent  on  system  nomenclature  and  by  47  percent  On  system  components. 


4.  All  of.  the  hybrid  aid  users  rated  their  aid  as  easier  to  use  end  understand  than 
technical  manuals,  compared  to  67  and  57  percent  of  the  DED  users,  In  ranking  JP  As,  the 
SS/BRC  was  ranged  first,  followed  fay  the  TS/IC,  die  UN,  and  the  D^D. 

Conclusions 

The  findings  suggest  that  #n  effective  EH3PA  has  a  directive  element  diet  allows  an 
inexperienced  technician  to  troubleshoot  accurately  ande  deductive  element  that  allows 
him  to  comprehend  die  troubleshooting  strategy  embodied  in  the  directive  element.  Ifhe 
addition  of  enrichment  supplies  the  rationale  foe  each  step  in  the  streets*.  Over  time, 
the  technician  pan  be  expect**  to  gain  an  understanding  of  ti*  troubleshooting  logic  of 
die  dteeetlve  element  and  transition  to  using  die  deductive  element  alone.  He  trill  then 
be  able  to  extend  lis  troubjefhooting  strategies  to  more  complex  symptoms  for  which 
th*?.  no  elds  and  rety  upon  conventional  functional  flow*  and  schematics  to 
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INTRODUCTION 


Problem 

Research  in  job  performance  aid  (JPA)  formats  for  troubleshooting  can  be  divided 
into  two  general  areas,  directive  (e.g.,  fully  proceduraiized  JPAs)  and  deductive  (e.g., 
functional  schematics).  Directive  formats  allow  novice  technicians  to  perform  on  the  job 
without  extensive  training  and  experience,  while  deductive  formats  allow  fully  trained  and 
experienced  technicians  to  continue  in  a  job  without  relying  on  memory  alone.  Neither 
format,  however,  is  satisfactory  for  an  individual  at  an  intermediate  level  of  skill. 

The  hybrid  JPA  presents  troubleshooting  information  in  both  directive  and  deductive 
formats.  The  intended  purpose  of  the  hybrid  JPA  is  to  allow  an  inexperienced  technician 
to  troubleshoot  a  given  set  of  symptoms  correctly  using  the  directive  part  of  the  hybrid. 
At  the  same  time,  by  examining  the  deductive  part  of  the  hybrid,  the  apprentice 
technician  can  begin  to  incorporate  standard  technical  data,  such  as  functional  flow 
diagrams  or  schematics,  in  his  troubleshooting  approach.  Over  time,  the  technician  should 
gradually  transition  from  the  directive  to  the  deductive  element  and  begin  to  develop 
troubleshooting  strategies. 

In  an  earlier  effort,  the  potential  of  "enriching"  the  hybrid  was  evaluated.  Enrich¬ 
ment  is  a  method  of  adding  job-relevant  information  that  should  facilitate  the  transition 
between  directive  and  deductive  formats  by  providing  the  technician  with  a  functional 
understanding  of  the  system.  Various  conceptual  formats  for  the  enriched  hybrid  JPA 
(EHJPA)  were  defined,  but  none  were  evaluated  in  an  operational  setting  to  determine 
whether  a  technician  could  transition  from  the  directive  to  the  deductive  or  if  the 
enrichment  information  contained  in  the  EHJPA  increased  the  technician's  functional 
understanding  of  the  system. 

In  addition  to  using  the  EHJPA  to  facilitate  an  inexperienced  technician's  trouble¬ 
shooting  abilities,  the  deductive  element  of  the  EHJPA  can  be  used  by  the  more 
experienced  technicians  for  troubleshooting.  In  the  instructional  setting,  the  EHJPA 
would  allow  instructors  to  point  out  to  the  students  the  relationship  between  the  directive 
elements  and  the  development  of  a  deductive  troubleshooting  strategy. 

Purpose 

The  purpcse  of  this  study  was  to  develop  and  evaluate  alternative  hybrid  JPA  formats 
to  determine  if  (1)  inexperienced  technicians  can  transition  to  a  deductive-only  JPA  after 
using  a  hybrid  JPA  to  facilitate  the  formation  of  troubleshooting  strategies  and  (2) 
placement  of  additional  information  (enrichment)  in  a  JPA  is  an  effective  technique  for 
increasing  technicians'  system  knowledge.  The  EHJPA  is  an  integral  component  of  the 
enlisted  personnel  individualized  career  system  (EPICS).  In  EPICS,  technicians  need 
troubleshooting  JPAs  to  supplement  the  formal  training  that  is  deferred  and  distributed 
over  an  individual's  career. 

Background 

Enlisted  Personnel  Individualized  Career  System 

Increasing  training  and  personnel  costs,  a  high  attrition  rate  with  its  accompanying 
loss  of  the  Navy's  investment,  a  decreasing  manpower  pool,  and  the  increase  in  techno¬ 
logical  complexity  of  systems  have  led  the  Navy  to  consider  alternate  career  systems 
(Blanchard  it  Laabs,  197S).  The  objectives  of  one  alternative,  EPICS  (Blanchard  it  Smiilie, 
1980),  are  to  (1)  get  the  individual  on  the  job  as  quickly  as  possible,  allowing  him  to 
contribute  to  ship's  work  w'  »h  minim  up  initial  investment  in  formal  training,  (2)  ensure 
that  the  system  provides  t  ~  Mvarv  ,.<ent  of  qualified,  career-oriented  personnel,  (3) 
improve  utilization  of  lesser-  *  , tit^s  personnel,  and  (9)  provide  ultimately  for  the 
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development  of  a  trained  career  force.  These  objectives  are  to  be  met  through  the  use  of 
JPAs,  deferred  formal  training,  transition  adaptation  training,  and  an  individualized 
career  advancement  structure. 

JPAs  in  EPICS 


Goff,  Schlesinger,  and  Parlog  (1969)  demonstrated  that  JPAs  are  an  effective  tool 
since  they  allow  those  individuals  with  little  formal  training,  as  well  as  those  of  lesser 
aptitude,  to  perform  productive  work.  JPAs  are  useful  within  EPICS  because  training  can 
be  deferred  until  it  has  been  determined  that  the  individual  has  the  ability  and  motivation 
to  take  advantage  of  a  formal  training  program.  In  addition,  deferment  gives  the  Navy 
time  to  determine  whether  such  a  training  investment  would  benefit  the  Navy  itself. 

Early  in  the  EPICS  career  path,  the  technician  performs  calibration,  remove  and 
replace  tasks,  and  preventive  maintenance  using  fully  proceduralized  JPAs  (FPJPAs). 
FPJPAs  are  comprehensive,  self-contained,  and  require  little  decision-making  by  the 
technician.  While  these  FPJPAs  meet  the  needs  of  a  novice  troubleshooter  by  providing 
the  necessary  guidance,  they  are  too  detailed  and  somewhat  demotivating  for  an 
experienced  troubleshooter.  As  a  technician  progresses  and  becomes  more  knowledgeable 
about  the  system  and  more  skillful  in  troubleshooting,  a  deductive  aid  (which  allows  him 
to  exercise  his  judgment  and  apply  his  expertise)  becomes  more  appropriate  and  useful. 
As  the  technician's  needs  change,  so  must  the  JPA. 

Hybrid  JPAs  and  Enrichment 

The  hybrid  aid  concept  first  emerged  from  a  review  of  types  of  troubleshooting  aids 
by  Post  and  Price  (1973),  which  concluded  that  one  or  the  other  of  the  terms  "directive" 
or  "deductive"  could  characterize  all  troubleshooting  presentation  techniques.  They 
proposed  that  a  single  JPA  in  which  the  directive  and  deductive  aid  presentation  formats 
appear  side-by-side  would  facilitate  the  development  of  troubleshooting  strategies  by 
inexperienced  technicians. 

The  enrichment  concept  was  developed  to  offset  negative  aspects  of  JPAs.  It  had 
been  found  that,  while  FPJPAs  could  produce  effective  and  efficient  maintenance  work, 
continued  use  of  such  aids  may  result  in  rote  task  performance,  little  learning,  and  job 
dissatisfaction  (Johnson,  Thomas,  &  Martin,  1977).  Enrichment  is  a  means  of  combining 
learning  materials  with  the  JPA  to  make  work  more  rewarding  and  to  promote  on-the-job 
learning.  Pulliam,  Goett,  and  Smith  (1979)  suggested  that  introducing  supplemental 
information  to  the  JPA  as  it  relates  to  the  task  at  hand  is  an  effective  way  to  impart 
additional  knowledge  to  the  user. 

Post  and  Smith  (1979)  developed  several  enriched  and  nonenriched  hybrid  aid  forms 
and  pilot  tested  them  on  untrained  subjects  for  their  ability  to  teach  novice  technicians  to 
troubleshoot  using  only  the  deductive  aid.  In  the  test,  subjects  initially  achieved  a  low 
solution  rate  using  only  the  deductive  aid.  After  several  learning  trials  with  the  hybrid, 
there  was  a  significant  increase  in  performance  when  once  again  only  the  deductive  aid 
was  used. 


APPROACH 


Experimental  JPA  Formats 

Four  experimental  JPA  formats  were  tested:  (1)  the  deductive  (DED)  JPA,  (2)  the 
unenriched  (UN)  HJPA,  (3)  the  task-specific/internally-cued  (TS/IC)  EHJPA,  and  (4)  the 
system-specific/basic  reference-cued  (SS/BRC)  EHJPA.  These  formats  are  illustrated  in 
Figures  1  through  4  and  are  described  in  the  following  paragraphs. 
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Figure  2.  Illustration  of  an  unenriched  (UN)  hybrid  job  performance  aid  (H3PA). 
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(S)  SI  >  «.S  VAC  IF  Cl 

<  S.S  VAC 


DVM  REQUIRED 

TEST  PASSES  IF  ANY  OF  THE  FOL¬ 
LOWING  IS  TRUE: 

(1)  C4  BETWEEN  1.4  AND  1.5  VAC 

(2)  84  >1.4  VAC  IF  C4  <  1.4  VAC 
(9)  84  <  1.5  VAC  IF  C4  >  1.4  VAC 

>  v  22V 

500  MV  P-P  (OSCILLOSCOPE) 

DVM  REQUIRED 
-9.91  TO  -4.11V 


TEST  POINT 


4A3A4PI-A4 


SIG  SEL  9  (ES) 

S1G  BEL  4  (E4) 
2NDLO  CONTROL 


PREDICTED  MBC 


ANT  90/90  (AS) 
MBC  LOCKUP 


1A2B2-R1/1A2B2-R4 

ROLL  0,  PITCH  90 
(A10,  All) 

WIND  ZERO  (AT) 


CMPTR  TE8T  INPUT  i 
(Al,  AS) 


<  ♦  ! 

>.92V 

EXTENDER  REQUIRED 
DVM  REQUIRED 
-8.2*  TO -9.98V 
(MOTE  1) 

DVM  REQUIRED 
-9.9  TO  -4.  IV 

COUNTER  REQUIRED 
15.759925  TO  90  952*25  ] 


DVM  REQUIRED 
1.55  TO  9.45V 

DVM  REQUIRED 
<  0.9  VAC 

DVM  REQUIRED 
1.90  TO  9. 90  VAC 


>  »29V 

10  VAC.  400  CP* 
1  VAC.  500  CPS 


b.  Deductive  element. 


Figure  2.  (Continued.) 
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a.  Directive  element. 

Figure  3.  Illustration  of  a  task-specific/internaUy-cued  (TS/IC)  enriched  hybrid  job 
performance  aid  (EHJPA). 
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b.  Deductive  element. 


Figure  3.  (Continued.) 
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b.  Deductive  element. 
Figure  4.  Continued. 
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Deductive  (PEP)  3PA 

Figure  1  shows  a  sample  of  a  DED  JPA  in  the  form  of  a  block  diagram.  Blocks 
represent  line  replaceable  units  (LRUs);  lines  and  arrows,  current  flow;  dashed  lines, 
dependencies;  and  heavy  dots,  test  points.  The  aid  lists  each  test  point  and  is 
accompanied  by  a  table  listing  the  test  points,  the  signals,  and  the  value  of  good  signals. 

The  DED  JPA  does  not  direct  the  technicians  as  to  what  troubleshooting  steps  to 
perform.  Rather,  the  user  must  be  a  decision  maker  and  must  "deduce"  the  most  efficient 
order  in  which  to  test  the  test  points.  Generally,  the  DED  JPA  is  best  used  by  more 
advanced  technicians  who  do  not  need  step-by-step  prompting. 

Unenriched  (UN)  HJPA 


Figure  2  shows  a  sample  of  an  UN  HJPA,  which  presents  deductive  and  directive 
troubleshooting  elements  side-by-side.  The  directive  element  (Figure  2.a)  is  a  decision 
tree.  The  steps  of  a  testing  procedure  are  contained  in  the  hexagonal  nodes.  In  each 
case,  a  good  or  bad  test  outcome  determines  the  ensuing  step.  The  directive  element  is 
not  a  fully  procedurized  JPA,  as  it  assumes  that  the  technician  knows  the  locations  of  the 
test  points,  is  familiar  with  the  use  of  test  equipment,  and  can  determine  good  and  bad 
outcomes.  However,  it  does  guide  him  through  a  troubleshooting  strategy. 

The  inexperienced  technician  can  perform  troubleshooting  immediately  by  following 
the  directive  portion  of  the  hybrid.  He  is  told  that  he  can  follow  the  step-by-step 
reasoning  behind  the  troubleshooting  procedure  by  studying  the  accompanying  deductive 
element  (Figure  2.b)  during  a  troubleshooting  task.  As  he  gains  experience,  he  should  be 
able  to  abandon  the  directive  element  and  rely  exclusively  on  the  deductive  element. 
Thereafter,  when  he  needs  help  or  wants  to  check  his  work,  he  can  refer  to  the  directive 
aid.  This  type  of  aid  contains  the  minimum  amount  of  information,  in  both  the  directive 
and  deductive  elements,  needed  by  a  technician  to  perform  his  job. 

Task-specific/Internally-cued  (TS/IC)  EHJPA 

Enrichment  is  information  added  to  HJPAs  to  (1)  point  out  to  the  less  insightful  users 
some  learning  opportunities  inherent  in  the  aid,  and/or  (2)  introduce  learning  possibilities 
not  normally  present  in  performing  a  maintenance  task  (e.g.,  theory-based  learning). 
When  the  enrichment  concerns  the  task  at  hand,  it  is  referred  to  as  task-specific  (TS). 
When  it  appears  on  the  aid,  it  is  referred  to  as  internally-cued  QC).  The  TS/IC  enrichment 
in  Figure  3  includes  the  bracketed  information  following  each  test  on  the  directive 
element  (Figure  3.a)  and  the  LRU  name  inside  the  blocks  on  the  deductive  element  (Figure 
3.b).  The  bracketed  information  on  the  directive  element  indicates  which  LRUs  on  the 
accompanying  deductive  element  are  cleared  or  suspected  as  a  result  of  the  test  outcome. 
The  purpose  of  this  enrichment  is  to  demonstrate  and  teach  a  troubleshooting  method  in 
which  technicians  rely  on  deductive  aids. 

System-specific/Basic-reference-cued  (SS/BRC)  EHJPA 

The  SS/BRC  format  is  similar  to  the  TS/IC.  The  difference  is  that  the  SS/BRC 
EHJPA  includes  paragraph  references  from  the  system  technical  manuals.  These 
references,  which  are  placed  above  or  below  the  LRU  block  of  the  deductive  element 
(Figure  *.b),  provide  general  (basic)  information  about  the  specific  system.  The  directive 
element  (Figure  4.a)  is  the  same  as  in  the  TS/IC. 
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Subjects 


Twenty  Navy  personnel,  having  a  mean  age  of  21.1  years  and  a  mean  experience  in 
electronics  of  6.2  months,  served  as  subjects.  They  were  classified  according  to  two 
levels  of  electronics  training— high  and  medium.  High  was  defined  as  having  completed 
"A"  school  in  electronics  and  the  organizational  classes  for  the  AN/A WG- 10  missile 
control  system  simulator.  Medium  was  defined  as  having  completed  "A"  school  only. 
Subjects  were  randomly  assigned  to  one  of  four  groups  according  to  the  type  of  aid  used— 
DED,  UN,  TS/IC,  and  SS/BRC. 

Equipment 

All  groups  were  tested  at  the  Naval  Air  Station,  Oceana,  Virginia  Beach,  Virginia  on 
the  AN/AWG-10  missile  control  system  simulator.  The  AN/AWG-10,  a  naval  airborne 
weapon  control  system,  is  basically  an  analog  system  with  some  digital  components  in  its 
computer. 

Procedure 


All  groups  were  given  a  short  introduction  and  orientation,  a  troubleshooting  pretask 
using  the  DED  format,  a  pretest  on  system  knowledge,  two  troubleshooting  tasks  using  one 
of  the  four  JPA  formats,  a  troubleshooting  posttask  using  the  DED  format,  a  posttest  on 
system  knowledge,  a  questionnaire/interview  on  user  acceptance,  and,  finally,  a  debriefing 
about  the  purpose  of  the  experiment.  These  events  are  described  below. 

1.  Introduction.  Participants  were  shown  the  layout  of  the  equipment  and  given  a 
general  explanation  of  the  tasks  they  were  to  perform. 

2.  Pre-  and  Posttests  on  System  Knowledge.  The  pre-  and  posttests  on  system 
knowledge  were  identical.  The  first  part  of  the  test  determined  the  subjects'  knowledge 
of  system  nomenclature  by  having  them  match  an  LRU  name  and  its  numerical 
designation.  The  second  part  measured  the  subjects'  technical  knowledge  of  particular 
components  of  the  system  used  during  the  troubleshooting  problems. 

3.  Troubleshooting  Tasks  and  Type  of  Aid  Used.  Each  subject  in  each  group 
attempted  to  fault-isolate  four  equipment  failures.  Each  task  required  the  subject  to  use 
the  aid,  locate  the  test  point,  set  up  the  test  equipment,  take  a  reading,  compare  the 
reading  to  the  proper  signal  value,  and  judge  whether  the  outcome  was  good  or  bad.  This 
process  was  repeated  until  the  subject  had  isolated  the  failed  LRU  or  exceeded  *0 
minutes.  Symptoms  related  to  the  failure  of  four  different  LRUs  were  used  as  the  four 
problems.  The  problems  were  counterbalanced  to  avoid  confusion  between  the  effects  of 
difficulty  of  task  and  learning  effects.  Table  1  illustrates  the  experimental  design. 

For  task  1  (pretask),  all  subjects  used  the  DED  format.  For  tasks  2  and  3,  subjects 
used  the  JPA  format  appropriate  for  their  assigned  group.  For  task  4  (the  posttask),  all 
groups  again  used  the  DED  format.  Subjects  were  assigned  to  the  DED  group  used  the 
DED  format  on  all  four  tasks. 

4.  Qucstionnaire/lnterview.  A  questionnaire  (see  appendix)  was  administered  in  two 
parts.  Part  I  concerned  subject  demographic  data,  amount  of  electronics  training.  Navy 
goals,  and  the  probability  of  the  subject's  pursuing  a  Navy  career.  Part  n  included  29 
items  on  learning,  enrichment,  motivation,  3 PA  attributes,  and  comparing  JPAs  to  current 
technical  materials.  Subjects  were  to  indicate,  on  a  3-point  scale,  how  much  they  agreed 
with  each  statement.  The  rating  scale  for  each  question  ranged  from  1  to  5,  where 
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Table  1 


Experimental  Design 


Group 

Subject 

Training 

Level 

Task 

Order 

Troubleshooting  Task  Number 
and  Type  of  Aid  Used 

1 

2 

3 

4 

DED 

1 

High 

Order  1 

DED 

DED 

DED 

DED 

2 

High 

Order  2 

DED 

DED 

DED 

DED 

3 

High 

Order  1 

DED 

DED 

DED 

DED 

4 

High 

Order  2 

DED 

DED 

DED 

DED 

5 

Medium 

Order  1 

DED 

DED 

DED 

DED 

6 

Medium 

Order  2 

DED 

DED 

DED 

DED 

UN 

1 

High 

Order  l 

DED 

UN 

UN 

DED 

2 

High 

Order  2 

DED 

UN 

UN 

DED 

3 

Medium 

Order  1 

DED 

UN 

UN 

DED 

4 

Medium 

Order  2 

DED 

UN 

UN 

DED 

TS/IC 

1 

High 

Order  1 

DED 

TS/IC 

TS/IC 

DED 

2 

High 

Order  2 

DED 

TS/IC 

TS/IC 

DED 

3 

High 

Order  1 

DED 

TS/IC 

TS/IC 

DED 

4 

High 

Order  2 

DED 

TS/IC 

TS/IC 

DED 

5 

Medium 

Order  1 

DED 

TS/IC 

TS/IC 

DED 

6 

Medium 

Order  2 

DED 

TS/IC 

TS/IC 

DED 

SS/BRC 

1 

High 

Order  1 

DED 

SS/BRC 

SS/BRC 

DED 

2 

High 

Order  2 

DED 

SS/BRC 

SS/BRC 

DED 

3 

Medium 

Order  1 

DED 

SS/BRC 

SS/BRC 

DED 

4 

High 

Order  2 

DED 

SS/BRC 

SS/BRC 

DED 

Legend:  DED  =  deductive  3PA. 

UN  =  unenriched  hybrid  JPA. 

TS/1C  =  task-specific/internally-cued  enriched  hybrid  3PA. 

SS/BRC  =  system-specific/basic-reference-cued  enriched  hybrid  3PA. 

1  =  agree  strongly,  2  =  agree,  3  =  disagree,  4  =  disagree  strongly,  and  5  =  don't  know  or  not 
applicable.  During  an  open-ended  interview,  subjects  were  able  to  express  their  opinions 
about  any  aspect  oi  the  aids  not  already  covered  in  the  questionnaire. 

Analyses 

Subjects  in  the  DED  group  were  used  as  a  control  group  and  compared  to  those  in  the 
UN,  TS/IC,  and  SS/BPC  groups  on  the  following  measures: 

1.  Fault-isolation  time. 

2.  Solution  rate  (i.e.,  the  proportion  of  problems  solved  correctly). 

3.  System  knowledge. 

The  questionnaire  was  used  to  obtain  qualitative  data  on  user  acceptance  of  the  four 
EH3PA  formats. 


RESULTS 


Fault-isolation  Time 


Table  2  shows,  by  group,  the  mean  fault-isolation  time  (in  minutes)  for  each  of  the 
four  troubleshooting  tasks.  Only  the  mean  fault-isolation  time  on  the  last  task  for  the 
TS/IC  group  was  significantly  faster  (p  <  .05)  than  the  DED  group  (t^  =  1.81).  There 

were  no  significant  differences  on  the  last  task  between  the  DED  and  SS/BRC  group,  the 
UN  and  DED  group,  or  between  any  of  the  hybrid  groups. 


Table  2 

Mean  Fault-isolation  Time  by  Group  and  Task  Number 


_ Time  (Minutes) _ 

Task  la  Task  4a 


Group 

N 

(Pretask) 

Task  2 

Task  3 

(Posttask) 

DED 

6 

33.1 

22.3 

23.1 

25.1 

UN 

4 

30.5 

17.7 

15.6 

16.7 

TS/IC 

6 

24.5 

13.6 

15.0 

14.8 

SS/BRC 

4 

18.1 

22.5 

17.8 

18.7 

aAll  groups  used  the  DED  format  on  tasks  1  and  4. 


Solution  Rate 


Table  3  gives  the  solution  rate  results  by  group  and  task  number.  After  an  initial  67 
percent,  the  DED  group  averaged  89  percent  fault  isolation  rate  on  the  last  three  tasks. 
For  all  hybrid  groups,  the  solution  rate  was  100  percent  after  the  initial  task,  in  which  the 
average  solution  rate  was  69  percent.  Note  that  on  every  problem  where  the  hybrid  was 
used  (i.e.,  Tasks  2  and  3),  the  failure  was  correctly  fault-isolated.  When  all  hybrid 
subjects  transitioned  back  to  the  deductive  aid  on  Task  4,  again  every  problem  was 
correctly  solved. 

The  difference  between  the  solution  rate  in  Task  4  and  Task  1  for  all  groups  was 
tested  with  the  Wilcoxon  Rank-Sum  test.  The  difference  was  significant  at  the  .05  level. 

System  Knowledge 

Part  I  of  the  test  of  the  enrichment  information  measured  the  subjects'  knowledge  of 
the  system  nomenclature  by  having  them  match  the  LRU  name  with  its  number.  This  type 
of  enrichment  was  provided  via  the  EHJPAs  to  the  TS/IC  and  SS/BRC  subjects  during 
Tasks  2  and  3.  The  SS/BRC  subjects  had  additional  learning  opportunities,  as  the  LRU 
name/number  was  not  only  on  their  aid  but  also  in  the  technical  manual  materials  that 
were  referenced  from  the  aid. 
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Table  3 


Solution  Rate  by  Group  and  Task  Number 


_ Solution  Rate  (Percent) _ 

Task  la  Task  4a 


Group 

N 

(Pretask) 

Task  2 

Task  3 

(Posttask) 

DED 

6 

67 

100 

83 

83 

UN 

4 

50 

100 

100 

100 

TS/IC 

6 

83 

100 

100 

100 

SS/BRC 

4 

75 

100 

100 

100 

aAll  groups  used  the  DED  format  on  these  tasks. 


Table  4  gives  (1)  the  mean  number  of  correct  responses  on  the  test  that  were  given 
before  and  after  the  troubleshooting  tasks,  (2)  the  mean  difference  score,  and  (3)  the 
percent  increase  for  the  groups.  Since  the  DED  and  UN  groups  were  never  provided  this 
enrichment,  their  sources  were  combined.  There  was  a  slight  decrease  in  the  combined 
DED  and  UN  scores  (guessing  was  allowed),  the  TS/IC  had  no  change,  and  the  SS/BRC 
group  had  a  38  percent  increase. 


Table  4 

Nomenclature  Test 


Mean  Number 

Correct  Responses 

Mean 

Percent 

Group 

N 

Pretest 

Posttest 

Difference  Score 

Change 

DED+UN 

10 

4.3 

4.1 

-0.2 

-5 

TS/IC 

5 

5.2 

5.2 

0.0 

0 

SS/BRC 

4 

5.3 

7.3 

4-2.0 

4-38 

The  Wilcoxon  test  indicated  that  there  was  no  significant  difference  at  the  0.10  level 
between  the  combined  DED-UN  and  the  TS/IC  groups'  mean  difference  scores.  There  was 
a  significant  difference  at  the  0.05  level  between  the  SS/BRC  and  the  combined  DED-UN 
groups'  mean  difference  scores  and  a  significant  difference  at  the  0.10  level  between  the 
TS/IC  and  the  SS/BRC  groups'  mean  difference  scores. 

Part  n  of  the  test  measured  the  subjects'  knowledge  of  the  function  of  system 
components  involved  in  the  troubleshooting  tasks.  During  troubleshooting,  only  the  group 
with  the  SS/BRC  had  the  opportunity  to  learn  component  functioning  by  referring  to 
paragraphs  in  the  technical  manual  that  were  pertinent  to  the  components  under  test. 
This  group  was  compared  to  the  UN  group,  the  only  group  using  an  HJPA  without  any 
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enrichment.  As  shown  in  Table  5,  use  of  the  technical  manual  material  while  trouble¬ 
shooting  resulted  in  a  47  percent  increase  in  the  SS/BRC  group  test  score.  The  UN  group 
subjects  did  not  have  the  opportunity  to  read  that  enrichment  and  their  scores  were 
virtually  unchanged.  The  Wilcoxon's  Rank-Sum  Test  indicated  at  the  0.05  level  that  the 
SS/BRC  difference  score  was  significaniy  higher  than  the  UN  group's  difference  score. 


Table  5 

Component  Function  Test 


Mean  Number 

Correct  Responses 

Mean 

Percent 

Group 

N 

Pretest 

Posttest 

Difference  Score 

Change 

UN 

4 

6.4 

5.8 

-0.6 

-9 

SS/BRC 

4 

4.8 

7.0 

+2.2 

+47 

User  Acceptance 

Of  the  hybrid  aid  users,  100  percent  rated  their  aids  as  easier  to  use  (question  1)  and 
easier  to  understand  (question  2)  than  technical  manuals.  The  DED  group  rated  their  aids 
less  favorably:  Only  67  percent  rated  it  easier  to  use  and  57  percent  rated  it  easier  to 
understand  than  technical  manuals.  However,  100  percent  of  all  subjects  indicated  that 
they  could  fault-isolate  faster  with  these  aids  than  with  technical  manuals  (question  25). 

Most  subjects  were  ambivalent  about  using  only  the  aids  to  perform  their  work 
(question  28),  but  all  of  them  indicated  that  they  would  become  better  technicians  more 

?uickly  by  using  the  aids  in  conjunction  with  the  technical  manuals  than  by  either  alone 
question  29). 

In  the  interview  phase,  the  subjects  were  shown  all  aid  forms.  Eighty-seven  percent 
of  those  who  were  asked  to  rank-order  their  JPA  preferences  ranked  the  SS/BRC  aid 
first,  the  TS/IC  aid  second,  the  UN  aid  third,  and  the  DED  aid  fourth.  The  overwhelming 
majority  of  subjects  preferred  the  SS/BRC  aid  because  of  the  amount  of  information  it 
contained  and  because  the  aid  referenced  the  technical  manual. 

The  deductive  aid  had  extremely  high  acceptance  among  the  instructor  personnel  who 
observed  and  helped  to  run  the  experiment.  They  particularly  liked  the  ability  of  the  aid 
to  give  the  technician  an  overall  picture— the  functional  layout  rather  than  the  tradi¬ 
tional  layout  keyed  to  individual  physical  pieces  of  equipment. 


DISCUSSION  AND  CONCLUSIONS 

Although  all  the  fault  isolation  times  for  the  hybrid  groups  were  faster  than  the  times 
for  the  deductive  control  group,  only  the  TS/IC  group  was  significantly  different  than  the 
DED  group.  When  subjects  used  the  TS/IC  EH3PA,  they  solved  the  two  test  problems 
(Task  2  and  Task  3)  58.7  percent  faster  than  did  the  subjects  using  the  DED  aid.  When 
both  groups  had  to  troubleshoot  using  only  the  DED  aid,  the  TS/IC  groups  isolated  the 
fault  69,6  percent  faster,  indicating  that  exposure  to  the  EHJPA  provided  technicians 
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with  the  functional  understanding  and  experiences  necessary  to  troubleshoot  using 
deductive  information  only.  Using  the  deductive  information  only,  subjects'  performance 
did  improve  from  Task  1  to  Task  4,  but  the  rate  of  improvement  was  less.  For  the  DED 
group,  the  improvement  rate  was  31.9  percent,  whereas  the  improvement  rate  for  the 
TS/IC  group  was  63.5  percent.  It  can  be  assumed  that  the  improvement  for  the  DED 
would  continue  to  improve  until  it  asymptotes  and  performance  is  equivalent  to  the  TS/IC 
group. 

Solution  rates  for  all  hybrid  groups  in  Tasks  2  and  3  were  100  percent.  Ragardless  of 
hybrid  aid,  subjects  could  solve  all  problems  when  only  the  deductive  information  was 
available:  All  the  subjects  who  had  used  the  hybrids  for  the  previous  problems  were  able 
to  solve  the  Task  4  problem.  Only  83  percent  of  the  DED  group  could  solve  the  Task  4 
problem. 

I 

In  acquiring  system  knowledge,  only  the  SS/BRC  group  did  better.  This  group  could 
name  components  and  describe  component  functions  better  after  having  used  the  EH3PA. 
This  improvement  may  be  the  result  of  the  subjects  reading  the  additional  technical 
material  since  there  was  no  improvement  for  the  TS/IC  group  on  the  nomenclature  test. 
Except  for  the  reference  cues  on  the  SS/BRC,  both  EHJPAs  were  the  same. 

The  SS/BRC  group  did  not  show  any  improvement  in  fault  isolation  times  from  Task  1 
to  Task  4.  It  appears  that  subjects  in  that  group  were  able  to  troubleshoot  quite  easily 
with  the  deductive  element  alone.  This  group,  compared  to  the  others,  had  the  largest 
ratio  of  highly  trained  technicians,  3  to  1.  Comparison  of  performance  times  for  Task  1 
shows  that  while  neither  the  UN  nor  TS/IC  groups  were  significantly  different  from  the 
DED  group,  the  SS/BRC  group  was.  Thus,  because  of  the  close  similarity  between  the 
TS/IC  and  SS/BRC  aids,  the  performance  of  the  TS/IC  group  provides  a  better  evaluation 
of  the  EH3PA. 

Opinion  data  showed  high  acceptance  of  the  hybrids.  The  data  indicate  that  100 
percent  of  those  using  hybrids  said  that  they  could  fault-isolate  faster  with  these  new 
experimental  aids  than  with  the  traditional,  more  deductive  technical  manual.  Further¬ 
more,  100  percent  of  those  using  hybrids  said  it  was  easier  to  use  and  understand  than 
existing  technical  material.  All  subjects  believed  that  they  would  become  better 
technicians  more  quickly  by  using  the  aids  in  conjunction  with  the  technical  manuals  than 
by  using  either  alone.  The  SS/BRC  was  rated  the  most  favorable  of  the  hybrid  aids  by  the 
inexperienced  technician. 

The  findings  suggest  that  an  effective  EH3PA  has  a  directive  element  that  allows 
accurate  troubleshooting  by  inexperienced  technicians.  The  deductive  element  allows  the 
technician  to  comprehend  the  troubleshooting  strategy  embodied  in  the  directive  element. 
The  addition  of  enrichment  supplies  the  rationale  for  each  step  in  the  strategy.  Over 
time,  the  technician  can  be  expected  to  gain  an  understanding  of  the  troubleshooting  logic 
of  the  directive  element  and  transition  to  using  the  deductive  element  alone.  Then  the 
technician  will  be  able  to  extend  his  troubleshooting  strategies  to  more  complex  sets  of 
symptoms,  for  which  there  are  no  aids,  and  rely  upon  conventional  functional  flows  and 
schematics  to  troubleshoot  the  problem. 


RECOMMENDATION 

Although  the  comparison  of  fault  isolation  times  indicates  better  performance  with 
the  TS/IC  EHJPA,  the  similarity  of  it  to  the  SS/BRC,  combined  with  the  results  of  both 
the  system  knowledge  test  and  user  acceptance  questionnaire/interview,  suggest  that  the 
SS/BRC  is  the  more  effective  EH3PA.  Therefore,  it  is  recommended  that  this  EHJPA 
format  be  developed  for  the  EPICS  technician,  who  will  be  required  to  provide  trouble¬ 
shooting  support  at  the  intermediate  level. 
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QUESTIONNAIRE 


Parti 

1.  Location  _ _  2.  Age  _  3.  Rate 

4.  Joined  Navy  in  (month)  _  (Year)  _ 

5.  Plan  to  leave  Navy  in  (month)  _  (Year)  _ 

6.  Circle  Navy  schools  attended: 

School  Months  in  School 

AFUN-P  _ 

BE&E  _ 

"A"  School  _ 

AFT A  _ 

AWG- 10  ”C"  School  _ 

Other 


Type  of  Work  Months 

7.  Electronic  experience  _ 

(Hobby,  job,  etc.)  _ 

8.  Your  goals  to  be  reached  while  in  Navy: 

a.  _ _ _ 

b.  _ _ _ ___ 

9.  Plan  to  make  Navy  a  career?  (circle  one)  Yes  Undecided  No 

10.  Probability  of  yourself  having  a  Navy  career,  (circle  one) 

0%  10%  20%  30%  40%  30%  60%  70%  80%  90%  100% 


A-l 


AS  =  Agree  strongly 

AS  =  Agree 

D  s  Disagree 

DS  =  Disagree  strongly 

N  =  Don't  know  or  not  applicable 

1.  I  found  the  materials  in  the  exercise  easier  to  use  than  what  I  now  use. 

AS  A  D  DS  N 

2.  Compared  to  other  Navy  technical  manual  materials  1  have  seen,  the  materials  used 
in  the  exercise  were  easier  to  understand. 

AS  A  D  DS  N 

3.  I  think  I  can  teach  mayself  a  lot  about  the  AWG-10  system  by  using  these  new 
materials. 

AS  A  D  DS  N 

4.  If  1  am  doing  electronic  troubleshooting,  I  like  to  get  the  job  done  as  quickly  as 
possible. 

AS  A  D  DS  N 

5.  I  hope  the  Navy  provides  me  with  technical  manuals  like  the  ones  used  in  this 
exercise. 

AS  A  D  DS  N 

6.  The  technical  manual  materials  used  were  OK  for  someone  just  starting  to  do 
troubleshooting  but  would  be  too  distracting  for  an  experienced  technician. 

AS  A  D  DS  N 

7.  The  materials  I  used  in  the  exercise  were  too  complicated. 

AS  A  D  DS  N 

g.  I  would  like  to  learn  as  much  about  electronics  as  I  can  so  that  I  can  advance  in  rate 
as  quickly  as  possible. 

AS  A  D  DS  N 

9.  Having  a  step-by-step  troubleshooting  procedure  on  one  page  of  a  technical  manual 
and  a  flow  diagram  on  the  facing  page  is  a  poor  arrangement  for  helping  the  technician  do 


10.  1  found  that  1  was  learning  a  lot  about  how  the  system  works  from  using  the  materials 
in  the  exercise. 

AS  A  D  DS  N 

11.  When  I  do  a  job  such  as  electronic  troubleshooting,  I  like  to  take  the  time  to  figure 
out  how  the  system  really  works. 

AS  A  D  DS  N 

12.  These  new  materials  helped  me  learn  more  about  the  system  as  it  was  telling  me 
what  steps  to  do. 

AS  A  D  DS  N 

13.  The  enrichment  helps  you  learn  what  this  system  does. 

AS  A  D  DS  N 

14.  The  enrichment  was  easy  to  understand. 

AS  A  D  DS  N 

15.  The  enrichment  made  the  task  more  interesting. 

AS  A  D  DS  N 

16.  Understanding  the  enrichment  will  make  me  a  better  technician  in  the  long  run. 

AS  A  D  DS  N 

17.  If  you  could  advance  as  quickly  as  your  knowledge  and  skills  develop,  these  new 
materials  would  be  helpful  in  obtaining  advancement. 

AS  A  D  DS  N 

18.  I  believe  it  to  be  a  practical  troubleshooting  aid. 

AS  A  D  DS  N 

19.  1  like  the  idea  of  two  different  adjacent  materials  that  vary  in  difficulty  and  detail. 

AS  A  D  DS  N 

20.  These  new  materials  appear  to  allow  a  person  to  rapidly  learn  his  job. 

AS  A  D  DS  N 

21.  Additional  training  is  very  important  to  me. 

AS  A  D  DS  N 


A- 3 


22.  I  think  a  person  without  formal  schooling  could  troubleshoot  most  of  the  system  using 
these  3PAs  if  he  had  some  PJE  (OJT)  from  peers. 

AS  A  D  DS  N 

23.  Even  if  I  am  not  going  to  have  a  Navy  career,  it  is  important  to  me  to  advance  as 
much  as  possible  before  separating  from  the  Navy. 

AS  A  D  DS  N 

24.  I  believe  other  troubleshooters  would  like  to  have  this  type  of  3PA. 

AS  A  D  DS  N 

25.  These  new  materials  allow  me  to  fault-isolate  the  problem  faster  than  by  using  the 
techniocal  manuals. 

AS  A  D  DS  N 

26.  As  an  inexperienced  technician,  I  can  learn  more  from  these  aids  than  from  technical 
manuals. 

AS  A  D  DS  N 

27.  It  is  easier  to  understand  the  LRU  relationships  using  these  aids  than  with  the 
technical  manuals. 

AS  A  D  DS  N 

28  I  would  prefer  to  use  these  new  materials  rather  than  what  I  use  now. 

AS  A  D  DS  N 

29.  I  can  become  a  better  AWG-10  technician  more  quickly  by  using  both  the  new 
materials  and  the  technical  manuals  than  1  can  using  the  technical  manuals  alone. 

AS  A  D  DS  N 

30.  The  opportunity  to  get  technical  training  can  be  a  factor  in  a  person's  decision  to  join 
the  Navy.  How  important  was  this  factor  in  your  own  decision?  (Express  your  answer  as  a 
percentage  compared  to  all  other  factors.) 

Technical  training  _ % 

All  other  factors  % 


DISTRIBUTION  LIST 

Deputy  Under  Secretary  of  Defense  for  Research  and  Engineering  (Research  and 
Advanced  Technology) 

Chief  of  Naval  Education  and  Training  (02),  (N-2),  (N-5) 

Chief  of  Naval  Technical  Training  (016) 

Commander  Naval  Air  Development  Center 
Commander  Naval  Weapons  Center 

Commander  David  W.  Taylor  Naval  Ship  Research  and  Development  Center 

Commander  Naval  Surface  Weapons  Center 

Commander  Naval  Ocean  Systems  Center 

Commander  Training  Command,  U.S.  Atlantic  Fleet 

Commander  Training  Command,  U.S.  Pacific  Fleet 

Commanding  Officer,  Fleet  Combat  Training  Center,  Atlantic 

Commanding  Officer,  Fleet  Combat  Training  Center,  Pacific 

Commanding  Officer,  Fleet  Training  Center,  San  Diego 

Commanding  Officer,  Naval  Underwater  Systems  Center 

Commanding  Officer,  Naval  Education  and  Training  Program  Development  Center  (Tech¬ 
nical  Library) (2) 

Commanding  Officer,  Naval  Education  and  Training  Support  Center,  Pacific 
Commanding  Officer,  Naval  Technical  Training  Center,  Corry  Station  (Code  10  IB) 
Commanding  Officer,  Naval  Training  Equipment  Center  (Technical  Library)  (5) 

Director,  Naval  Education  and  Training  Program  Development  Center  Detachment,  Great 
Lakes 

Director,  Training  Analysis  and  Evaluation  Group  (TAEG) 

Chief,  Army  Research  Institute  Field  Unit— USAREUR  (Library) 

Commander,  Air  Force  Human  Resources  Laboratory,  Lowry  Air  Force  Base  (Technical 
Training  Branch) 

Commanding  Officer,  U.S.  Coast  Guard  Research  and  Development  Center,  Avery  Point 
Commanding  Officer,  U.S.  Coast  Guard  Training  Center,  Alameda 
Defense  Technical  Information  Center  (DDA)  (12) 


